Local and global explanations of agent behavior: Integrating strategy summaries with saliency maps

With advances in reinforcement learning (RL), agents are now being developed high-stakes application domains such as healthcare and transportation. Explaining the behavior of these is challenging, environments which they act have large state spaces, their decision-making can be affected by delayed rewards, making it difficult to analyze behavior. To address this problem, several approaches been developed. Some attempt convey $\textit{global}$ agent, describing actions takes different states. Other devised $\textit{local}$ explanations provide information regarding agent's a particular state. In paper, we combine global local explanation methods, evaluate joint separate contributions, providing (to best our knowledge) first user study combined for RL agents. Specifically, augment strategy summaries that extract important trajectories states from simulations agent with saliency maps show what attends to. Our results choice include summary (global information) strongly affects people's understanding agents: participants shown included significantly outperformed who were presented randomly set chosen world-states. We find mixed respect augmenting demonstrations (local information), addition did not improve performance most cases. However, do some evidence help users better understand relies on its decision making, suggesting avenues future work further